Skip to content

Conversation

@Anshul-creator
Copy link
Contributor

@Anshul-creator Anshul-creator commented Oct 28, 2025

What is the purpose of the change?

This PR fixes order-dependent flakiness (OD-Vic) in ExtensionLoaderTest.testInjectExtension

This test fails under randomized test execution orders due to multiple DemoImpl instances being registered in the shared ScopeBeanFactory of ApplicationModel.

Root Cause

testInjectExtension and related tests (testGetOrDefaultExtension) both interacted with the same shared ScopeBeanFactory.
Because bean registration was not idempotent, repeated or parallel test execution sometimes resulted in duplicate DemoImpl entries, leading to inconsistent injection results or assertion failures.

What's Changed

  • Added defensive logic to check existing DemoImpl beans before registering a new one.
  • Made registration idempotent by registering only one instance (demo-test-singleton) if none exists.
  • Ensured subsequent tests reuse the same canonical bean reference.
  • Updated testGetOrDefaultExtension to follow the same consistent setup.

These changes eliminate shared-state interference and make the test deterministic under any order or random seed.

Verification

You can try running the loop below on pre-fix and post-fix code:

fail=0; total=0
for s in 7138 7351 7892 11170 15857 15869 17721 20421 21699 22631 24059 29905; do
  echo "SEED=$s"
  total=$((total+1))
  SEED=$s ./mvnw -pl dubbo-common -DfailIfNoTests=true -DforkCount=1 -DreuseForks=false \
    -Dsurefire.runOrder=random \
    -Djunit.jupiter.testclass.order.default=org.junit.jupiter.api.ClassOrderer\$Random \
    -Djunit.jupiter.testmethod.order.default=org.junit.jupiter.api.MethodOrderer\$Random \
    -Djunit.jupiter.testclass.order.random.seed="$s" \
    -Djunit.jupiter.testmethod.order.random.seed="$s" \
    -Dtest=org.apache.dubbo.common.extension.ExtensionLoaderTest test \
    >/dev/null || { echo "** FAILED on SEED=$s **"; fail=$((fail+1)); }
done
echo
echo "===== SUMMARY ====="
echo "Total seeds: $total"
echo "Failed:      $fail"
echo "Passed:      $((total - fail))"
echo "==================="

The seeds were found by using a script I developed locally. Sometimes not all seeds fail on execution, but some will always fail on pre-fix code. However, on post-fix code none of the seeds fail.

Checklist

  • Make sure there is a GitHub_issue field for the change.
  • Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
  • Write necessary unit-test to verify your logic correction. If the new feature or significant change is committed, please remember to add sample in dubbo samples project.
  • Make sure gitHub actions can pass. Why the workflow is failing and how to fix it?

@Anshul-creator Anshul-creator changed the title Fix: Stabilize ExtensionLoaderTest.testInjectExtension (OD-Vic flakiness due to duplicate DemoImpl registration) Fix: Stabilize ExtensionLoaderTest.testInjectExtension Oct 28, 2025
@codecov-commenter
Copy link

codecov-commenter commented Oct 28, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 60.76%. Comparing base (3720471) to head (fc9416c).

Additional details and impacted files
@@             Coverage Diff              @@
##                3.3   #15744      +/-   ##
============================================
- Coverage     60.76%   60.76%   -0.01%     
+ Complexity    11727    11720       -7     
============================================
  Files          1938     1938              
  Lines         88637    88637              
  Branches      13380    13380              
============================================
- Hits          53864    53857       -7     
- Misses        29245    29247       +2     
- Partials       5528     5533       +5     
Flag Coverage Δ
integration-tests-java21 32.37% <ø> (+0.04%) ⬆️
integration-tests-java8 32.38% <ø> (-0.16%) ⬇️
samples-tests-java21 32.09% <ø> (-0.03%) ⬇️
samples-tests-java8 29.75% <ø> (-0.08%) ⬇️
unit-tests-java11 59.03% <ø> (+0.01%) ⬆️
unit-tests-java17 58.50% <ø> (-0.02%) ⬇️
unit-tests-java21 58.53% <ø> (+0.03%) ⬆️
unit-tests-java25 58.48% <ø> (+0.02%) ⬆️
unit-tests-java8 59.01% <ø> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@zrlw zrlw added the type/enhancement Everything related with code enhancement or performance label Nov 4, 2025
@zrlw
Copy link
Contributor

zrlw commented Nov 4, 2025

fail=0; total=0
for s in 7138 7351 7892 11170 15857 15869 17721 20421 21699 22631 24059 29905; do
  echo "SEED=$s"
  total=$((total+1))
  SEED=$s ./mvnw -pl dubbo-common -DfailIfNoTests=true -DforkCount=1 -DreuseForks=false \
    -Dsurefire.runOrder=random \
    -Djunit.jupiter.testclass.order.default=org.junit.jupiter.api.ClassOrderer\$Random \
    -Djunit.jupiter.testmethod.order.default=org.junit.jupiter.api.MethodOrderer\$Random \
    -Djunit.jupiter.testclass.order.random.seed="$s" \
    -Djunit.jupiter.testmethod.order.random.seed="$s" \
    -Dtest=org.apache.dubbo.common.extension.ExtensionLoaderTest test \
    >/dev/null || { echo "** FAILED on SEED=$s **"; fail=$((fail+1)); }
done
echo
echo "===== SUMMARY ====="
echo "Total seeds: $total"
echo "Failed:      $fail"
echo "Passed:      $((total - fail))"
echo "==================="

It might be better that adding an unit test to prove this PR.

@Anshul-creator
Copy link
Contributor Author

Thanks for the review!

I believe there might be a small misunderstanding...this PR doesn’t introduce new behavior that needs its own unit test. It fixes order-dependent flakiness in the existing test ExtensionLoaderTest.testInjectExtension

Before this change, the ExtensionLoader classes' tests, particularly testGetOrDefaultExtension (polluter) and testInjectExtension (victim) interfered with each other via shared ScopeBeanFactory state. Running the shell command below exposes this flakiness:

fail=0; total=0
for s in 7138 7351 7892 11170 15857 15869 17721 20421 21699 22631 24059 29905; do
  echo "SEED=$s"
  total=$((total+1))
  SEED=$s ./mvnw -pl dubbo-common -DfailIfNoTests=true -DforkCount=1 -DreuseForks=false \
    -Dsurefire.runOrder=random \
    -Djunit.jupiter.testclass.order.default=org.junit.jupiter.api.ClassOrderer\$Random \
    -Djunit.jupiter.testmethod.order.default=org.junit.jupiter.api.MethodOrderer\$Random \
    -Djunit.jupiter.testclass.order.random.seed="$s" \
    -Djunit.jupiter.testmethod.order.random.seed="$s" \
    -Dtest=org.apache.dubbo.common.extension.ExtensionLoaderTest test \
    >/dev/null || { echo "** FAILED on SEED=$s **"; fail=$((fail+1)); }
done
echo
echo "===== SUMMARY ====="
echo "Total seeds: $total"
echo "Failed:      $fail"
echo "Passed:      $((total - fail))"
echo "==================="

These seeds reliably reproduce the flakiness on the pre-fix code.

I attempted to simplify this to a single command (e.g., running the polluter → victim pair deterministically), but Surefire didn’t consistently preserve the inter-test interference needed to trigger the bug. The above loop was therefore the most reliable way to demonstrate the issue.

After this fix, none of these seeds fail. The test is now stable under any random execution order.

@zrlw
Copy link
Contributor

zrlw commented Nov 6, 2025

i mean you could write an junit test by calling java Process to run your script and assert its result.

@Anshul-creator
Copy link
Contributor Author

I will look into this and add a comment when I made changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type/enhancement Everything related with code enhancement or performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants